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There are three types of contexts subject to 
evaluation of student writing; the textual context that influences 
grammatical acceptability and the rhetorical effectiveness of a 
sentence; the coded context or cultural constraints such as generic 
and stylistic conventions; and pragmatic contexts that unite form, 
function, and setting in a developmental relationship. Current 
evaluation systems can be categorized according to which context they 
address. Text-oriented systems view the text itself as entirely 
sufficient to express meaning and to reveal author intent. 
Code-oriented systems emphasize the role that conventions and rules 
play in text interpretation and evaluation, and, although no fully 
developed system for evaluating pragmatic contexts exists, some 
methods include reader-response thinking and others use protocol 
analysis as a means for making judgments about the text as it 
develops in the writer's mind. What is needed, however, are systems 
with process-oriented contexts, especially since most evaluation 
systems have been product-bound. Two possible approaches that can be 
taken for the development of process-oriented evaluation systems are 
(1) multiple draft measurements that would analyze the nature and 
quality of the changes made between drafts and would make 
"appropriate change" a criterion of evaluation; and (2) writer-reader 
protocols that would feature articulated response as a criterion of 
evaluation and assess the stated intent of the author vis-a-vis the 
effect the author's work actually has on its audience. (HOD) 
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1 a | In Toward a Speech Act Theory of Literary Discourse , Mary Louise 

Pratt uses speech act theory to argue against the New Critical approach 
to literary criticism, that is, against an approach defined exclusively 
by textual considerations. Citing observations by sociolinguist William 
Labov, Pratt draws parallels between the devices of "natural narrative" 
embedded in speech acts and those elements informing the narrative of 
literary discourse. In so doing, she suggests how speech act theory can 
provide the groundwork for a context-dependent theory of literature. 

•Whether or not we, as teachers of writing, subscribe to Pratt's 
critique of New Criticism as a way of evaluating literary works, we can 
admit that, for the most part, we approach or evaluate student papers 
textually, not contextually. Yet these papers are the products of a 
process which involves a wide range of contexts, including personal world 
views and social milieu — the private and public worlds of our student 
writers. This paper, then, explores the question: how can we (or can 
we) go beyond the text and evaluate student writing contextually? 

In addressing this question, I will discuss types of contexts 
subject to evaluation, inventory a number of current evaluation systems 
as they approach one or more of these contexts, and conclude by 
suggesting avenues of development for contextual evaluation. 
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CONTEXTS 

We may begin by positing three types of contexts subject to evalua- 
tion: textual contexts, cor^d contexts, and pragmatic contexts. These 
categories, while not all-inclusive, will give us an operational base 
for discussion. 
Textual Contexts 

The text, of course, creates its own contexts. These textual con- 
texts may, for example, Influence grammatical acceptability. Consider 
the following: 

1. My peers. 

2. Become a computer programmer? 

3. Like fighting for the hiring of more full-time faculty and making 
some attempt to end exploitation rather than Increase it? 

All fragments. Yet rendered correct by their textual context. (For that 
context see '"Memoirs and Confessions of a Part-time Lecturer," College 
English, January 1982, pp. 36 & 39.) 

Textual context also Influences the rhetorical effectiveness of a 
sentence. This is especially true with "non-initial sysenatlc 
structures" containing allusions or references to textual situation:* 

1. And then there is feedback. (College Composition and Communica- 
tion , December 1980, p. 397.) 

2. So the intellectual climate conducive to this change has been 
developing for more than two decades. (College Composition 
and Communication , February 1982, p. 81.) 

3. Such separation is not always easy. (College Composition and 
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Communication , December 1981, p. 401.) 
Each of the above sentences achieves rhetorical effectiveness only 
when read within the larger framework of its respective text. As E. D. 
Hirsch, Jr. asserts in The Philosophy of Composition , "Any sentence in 
isolation is semantically uncertain, because it can mean whatever it 
might mean in an indefinite number of actual uses." Hirsch concludes it 
should therefore be our main business as composition teachers to show 
students how to make their writing "self-contextual." 
Coded Contexts 

Texts exist within contexts. Some of these contexts can be said to 
be coded; that is, they are systematized in some way so that they can serve 
as signs or principles for the participants in a communication situation. 

Whether public or private, coded contexts inform both oral and 
written discourse. Speech act theorists, for example, see conversation 
itself as rule (or code) governed behavior. Theorist H. P. Grice uses 
four sets of maxims to describe conversational rules: 

1. maxims of quantity: provide enough information, but not too much 

2. maxims of quality: be truthful, have evidence 

3. maxims of relation: be relevant 

3 

4. maxims of manner: be orderly, clear, brief. 

If, for example, a speaker does not provide enough information for the 
current purposes of the exchange, he or she violates the maxim of quantity 
and thus may alienate listeners. In other words, conversations follow 
certain codes which are recognized by both speaker and listener. 

The same norms seem to govern written discourse. Generally, 
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writers arc expected to meet readers' expectations by providing adequate 
information or detail , by presenting generalizations which are "true" or 
tenable, by providing a focused and coherent discussion, and by being clear 
and concise. If, for instance, a writer fails to provide enough specific 
detail, he or she violates the maxim of quantity. In so doing, this 
writer may fail to convey the intended meaning to the audience. 

Beyond establishing such general constaints, codes can feature 
specialized expectations. In his work Language in the Inner City , William 
Labov studies how specialized codes can govern oral discourse. He, for 

example, describes sophisticated rules which seem to govern the ritual 

4 

insults and natural narratives of inner city speakers. Codes governing 
written discourse can also be specialized, ranging from those which define 
the structure of a Ciceronian argument to those which govern the organi- 
zation of a particular business letter, such as a negative claim 
adjustment. Generic and stylistic conventions, in other words, qualify 
as coded contexts. 

In Semiotics and Interpretation , Robert Scholes suggests an even 
broader definition of coded contexts. He sees the codes of both oral and 
written discourse as part of a larger pattern of cultural constraints 
These cultural constraints themselves serve as coded contexts for 
writers. In this regard, Roland Barthes has observed that writers are 
influenced in their writing "by pressures beyond their immediate aware- 
ness, such as the broad cultural expectations and influences of history 
and tradition— frames and schemata of much more inclusive proportions" 
than those of a writer's immediate cultural conventions or personal 
background 
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Personal background is nevertheless important. It contributes to a 
writer's set of assumptions about the world which informs a private coded 
context. This private codeu context, along with the more public contexts of 
generic conventions and cultural constraints, influences how a writer 
thinks and writes. Although it is beyond the scope of this paper to explore 
in detail how private codes influence a writer's thinking and writing, it 
can be shown here how subjective elements color certain tasks a writer 
faces. For example, when a writer engages in generative procedure, he or 
she necessarily invokes a set of assumptions about the world, a personal 
postulate system to help answer not only questions such as "How well 
does my topic reflect my social and/or ethical beliefs?" and "Why. does 
the topic seem relevant to me?", but also those such as "What is the 
audience in relationship to me?" (hostile, objective). "Who is the 
audience in relationship to me?" (teacher, peer), and "How well does my 
audience know my subject?" 7 The writer's personal postulate system thus 
forms a subjective assessment of the rhetorical situation. The postulate 
system informs the set of values the writer projects on subject and 
audience, and influences author intent. Through composition this system, 
or a portion of it, is made public. 8 This is not to say. however, that 
the private codes of a writer are ever perfectly known. 
Pra gmatic Contexts 

Texts also exist within contexts which remain process-oriented in 
nature. Pragmalinguistics studies such contexts in oral discourse. Prag- 
malinguistics or pragmatics is concerned with the relation that "unites 
(a) linguistic form and (b) the communicative functions that these forms 
are capable of serving, with (c) the settings in which those linguistic 
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forms can have those communicative functions." 

Linguist Charles J. Fillmore, for example, is interested in how the text 
"develops in response to ongoing events," both external and internal, and in 
how formal properties of texts can be related both to what the participants 
(the producers and interpreters of texts) are doing and what they are 
mentally experiencing/ 1 ^ Fillmore promotes discourse analysis which 
emphasizes the text's development in time, both from the communicator's and 
the audience's standpoint. Fillmore stresses a context which unites form, 
function, and setting in a developmental relationship. 

Linda Flower and John Hayes in "A Cognitive Process Theory of Writing" 
share Fillmore's developmental emphasis. In their model, "the major units 
of analysis are elementary mental processes" which have a "hierarchical 
structure." Flower and Hayes are interested in capturing "what is going on 
in the writer's mind during the act of composing itself ♦ "** In terms of 
interpretation, however, they restrict their attention to the writer's 
perception of his or her evolving text; they do not explore the audience's 
unfolding response to that same text. 

EVALUATION SYSTEMS 

Evaluation systems do exist which can deal with the aforementioned 

contexts. Indeed, current evaluation systems can be categorized according 

to which context they address. 

Evaluating Textual Contexts 

Text-oriented or "new critical" systems view the text itself as 

12 

entirely sufficient to express meaning and to reveal author intent. For 
example, E. D. Hirsch advocates judging texts according to their 
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"intrinsic quality of presentation." Hirsch argues that this quality 

can be defined as the text's "relative readability." Central principles 

of readability include speedy resolution of semantic-syntactic ambiguity 

and speedy fulfillment of semantic-syntactic expectations. Although Hirsch 

acknowledges that extrinsic categories of assessment, including a reader's 

judgment concerning an author 9 s intentions, do enter into evaluation, he 

prefers to evaluate a writer's work solely according to its 

intrinsic quality of presentation and its correctness. Nancy Sommers in 

"Responding to Student Writing" reflects this orientation in her call for 

text-specific criteria of evaluation. Sommers decries evaluative comments 

which can be "rubber-stamped" from text to text. She believes evaluation 

14 

should entail text-specific recommendations for revision. 
Evaluating Coded Contexts 

Code-oriented systems emphasize the role conventions and rules 
play in text interpretation and evaluation. In terms of interpretation, 
these systems assume that knowing codes helps the reader understand the 
text in situations where the writer says one thing but means something else, 
as is the case with "exaggeration, understatement, hints, irony, and all 
forms of figurative language— in particular, metaphor. As stated by 
J. L. Morgan, our ability to understand what is being said or written 
depends not only on "conventions of language, that jointly give rise to 
the literal meanings of sentences," but also on "conventions about language 
that govern the use of sentences, with their literal meanings, for certain 
purposes" and that allow us to "somehow infer" those purposes or intentions. 

In terms of evaluation, code-oriented systems can be divided into 
those which evaluate a text according to public codes and those which 
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emphasize the influence of private codes. "Semiotic systems 11 focus on public 

codes. Lloyd- Jones 1 Primary Trait Scoring System is an example of this type, 

because it looks to norms invoked by rhetorical contexts for its evaluation 

criteria. 1 ^ The Wilkenson Model, as adapted by Marilyn Stemglass, is 

another example. This Model assumes identifiable progress points within 

each of its four scales. The four scales of development are Stylistic 

Measures, Affective Measures, Cognitive Measures, and Moral Measures* The 

"moral measures 11 scale, for instance, evaluates how and to what degree a writer 

18 

has "internalized the morality of the culture." In both Lloyd-Jones and 
Wilkenson, certain coded contexts, whether rhetorical or cultural, provide 
benchmarks for evaluation. This is the case even though the Wilkenson 
Model is meant to be developmental and in this respect process-oriented. 

"Intentional systems" explore the private codes a writer uses in a 
message. These systems handle author intent, error-analysis, and self- 
evaluation. Mary Louise Pratt proposes a method of evaluation which analyzes 
a text according to how its author purposely follows or flouts generic and 
stylistic conventions. Pratt's system could be adapted to analyzing student 
essays where the writer claims to or attempts to use a particular 
convention. In a different aspect, error-analysis examines surface structures 
to discover a writer's underlying and often idiosyncratic rules for composing. 
It is probably the most familiar evaluation system in this vein. Self- 
evaluation systems straddle the dine between being code-oriented and process- 
oriented. In "How Writers Evaluate Their Own Writing," Susan Miller 
stresses the importance of self-evaluation according to subjective standards; 
in "Teaching the Other Self: The Writer's First Reader," Donald Murray 
posits the existence of an "other self" which monitors a writer's composing 
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activity. Both Miller and Murray see the private codes that writers 

establish for themselves as crucial for the writers 1 evaluating their own 
19 

work. A teacher can incorporate such self-evaluation into his or her 
grading by Mving students articulate how well the finished product fulfills 
their intentions or measures up to their expectations. Self-evaluation can 
also monitor the writer's work-in-progress and thus be part of a process^- 
oriented system. 
Evaluating Pragmatic Contexts 

No current process-oriented evaluation system clearly examines and 
evaluates texts as they develop for both writer and readar in the way 
Fillmore's work on developing response suggests is necessary. Flower 
and Hayes focus on the writer's processes and propose protocol analysis as 
a means for making judgments about the text as it develops in the writer's 
mind. In this method, a writer orally records responses to the 
emerging text. Other process-oriented systems examine how the text unfolds 
in the reader's mind. Although no fully developed system of reader-response 
criticism for student essays exists, Sommer's method includes some reader- 
response thinking. "In a first or second draft," she writes, 'W need to 

respond as any reader would, registering questions, reflecting befuddlement, 

•i20 

and noting places where we are puzzled about the meaning of the text. 

Fillmore's own approach to discourse analysis in part resembles that of 

reader-response critic Stanley Fish who "rivets attention on the sequence 

of decisions, revisions, anticipations, reversals, and recoveries that 

the reader performs as he [or she] negotiates the text sentence by sentence 

21 

and phrase by phrase." 

Fillmore, however, would join reader- response with writer- response 
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to achieve a more comprehensive method of assessment. As has been indicated • 
such a comprehensive process-oriented system which would consider both 
writer and reader remains without a prototype. 



An overview of evaluation systems suggests that we have been able to 
evaluate student writing contextually if the contexts involved have been 
somehow product-bound. We need, however, to develop further those systems 
concerned with process-oriented contexts. We especially lack a system 
which evaluates the text as it evolves for both writer and reader. 
Below are two of many avenues we can take in developing process-oriented 
evaluation systems. 

1. Multiple Draft Measurements. Sommers and others recommend teachers 
examine more than one draft per assignment. However , this multiple- 
draft approach as it i3 currently conceived still measures each 
draft as a product. To become process-oriented » a multiple-draft 
approach would have to analyze the nature and quality of the 
changes made between drafts and would have to make appropriate 
change a criterion of evaluation. 

2. Writer-Reader Protocols. Taped writer-reader protocols could 
evaluate draft s-in-progress or final products. Writers and 
readers alike could orally respond to what was being or had been 
written. Their response could proceed sentence-by-sentence or 
paragraph-by-paragraph. Writers and readers could also 
specifically respond to changes made between drafts. Analysis of 
these protocols could highlight similarities/differences between 
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the responses of writer and reader or between the reactions of 
two or more separate readers. Protocols would feature articu- 
lated response as a criterion of evaluation. They also could 
feature an assessment of the stated intent of the author vis a 
vis the effect the author f s work actually had on its audience. 
The absence of a good process-oriented system of evaluation to date 
may be, in the end, an indication of theoretical rather than pedagogical 
shortfall. Perhaps one reason for product-oriented systems dominating our 
current pedagogy is that the contexts they judge are more or less based on 
transmission models of communication. In his book Cognition, Convention 
and Communication (1980), Mark H. Bickhard distinguishes between trans- 
mission and transformational models of communication. According to Bickhard, 
transmission models, which assume that "communication consists fundamentally 

of the transmission of the encodings of underlying knowledge, " are essen- 

22 

tially product-oriented. Even pragmatic contexts, which represent a 
movement toward a more process-oriented or transformational approach, 
feature a process which "does not go beyond the act-that-transmlts." 

Bickhard views product-oriented or transmission models as untenable. 
He proposes instead transformational models which view communication "as 
a form of interactive transformation" where what is being changed is 
neither the understanding of the audience nor the perspective of the commu- 
nicator. Success or failure of communication does not depend on the writer's 
effecting the intended response in the reader's mind. In a transformational 
system, the object of communication itself becomes a social entity." It 

is "a commonality of understanding among the participants . . . a social 

23 

definition of the situation." A transformational model features interaction 
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within the social environment as its crucial context. 

Whether or not Bickhard's model can be fruitfully applied to composi- 
tion remains an issue open to study and debate. His emphasis on the inter* 
active nature of the communication process, however, should provide insights 
into the process-oriented contexts which influence composing and, in 
turn, insights into the evaluation systems which address these contexts* 
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